1. Field of the Invention
The present invention generally relates to a temporal smoothing apparatus and method for synthesizing an intermediate image, and more specifically, to a temporal smoothing apparatus and method capable of adaptively adjusting contribution degrees of a previous image and a present image in accordance with the types of an image to synthesize an intermediate image.
2. Description of the Related Art
To present more realistic images, image display systems such as televisions require three-dimensional image rendering devices and image processors. Three-dimensional display technologies can be largely classified into optical approaches and image-processing approaches. Specifically, the three-dimensional image processing field has been developed focusing on compression, decompression, transmission, and synthesis of image signals. Currently, a stereo (two viewpoints) technology to be applied to the simplest three-dimensional data terminals is employed in the game industry or the medical industry, and a three-dimensional broadcasting system employing the stereo technology has been realized in Japan and other countries.
FIG. 1 is a diagram schematically illustrating a three-dimensional image processing method. The three-dimensional image processing method includes: taking images of an object from various viewpoints and compressing the images into a predetermined image format; receiving and decompressing the compressed images and synthesizing an intermediate image from the images of various viewpoints, converting the transmitted images taken from various viewpoints and the synthesized intermediate image into a format which can be processed by a display apparatus; and finally providing a three-dimensional image to a, viewer by combining and displaying the images taken from various viewpoints.
A multi viewpoint three-dimensional image display apparatus requires images acquired from many viewpoints, but only images acquired from limited viewpoints are used due to band restriction. For this reason, the viewer feels as if he/she watches the images from a fixed specific position. In order to solve this problem, an intermediate image synthesizing technology to synthesize an intermediate image from images acquired by multi viewpoint cameras is employed. Such an intermediate image synthesizing technology is described in detail in the paper, “An Object-based System for Stereoscopic Viewpoint Synthesis” by J_R. Ohm, IEEE Trans. Circuit and System for Video Tech. Vol. 7, No. 5, pp. 801-811 October 1997, and the paper, “Robust Quadtree-based Disparity Estimation for the Reconstruction of Intermediate Stereoscopic Images” by A. Mancini, San Jose, USA SPIE, January 1998.
When the intermediate image is synthesized using the intermediate image synthesizing technology, a parallax estimation error between frames can occur due to noise or minute camera jerks. The parallax estimation error can occur in edges of the object in the synthesized intermediate image, and is observed as a flickering phenomenon by the viewer. Temporal smoothing technologies are used for removing the flickering phenomenon.
FIG. 2 is a diagram illustrating a conventional temporal smoothing technology. An intermediate image is generated from two images, i.e., a left image and a right image. A left disparity vector estimator 210 generates left disparity vectors 221 and 222 corresponding to a left image LIT-1 211 at a previous time T−1 and a left image LIT 212 at a present time T. A disparity vector indicates a range in which a specific image block in a target image is moved with respect to a reference image. That is, the left disparity vector indicates a range in which an image block of a predetermined size in the left image is moved with respect to the right image.
A left temporal smoothing unit 220 generates a smoothed disparity vector 231 from the generated previous and present left disparity vectors 221 and 222. The smoothed disparity vector is a value obtained by adding the present disparity vector to the previous disparity vector, and indicates an accumulated displacement value in which the specific image block in the left image is moved with respect to the right image. A left disparity vector (DV) densifying unit 230 generates a left pixel disparity vector 232 by scaling every pixel of the generated left disparity vector.
The right disparity vector is generated in the same manner as the left disparity vector. That is, a pixel disparity vector 237 of the right image is generated with reference to the right image through a right disparity vector estimator 215, a right temporal smoothing unit 225, and a right disparity vector densifying unit 235. The right disparity vector estimator 215 generates right disparity vectors 226 and 227 corresponding to a right image RIt-1 217 at a previous time T−1 and a right image RIT 216 at a present time T. The right temporal smoothing unit 225 generates a smoothed disparity vector 236 from the generated previous and present right disparity vectors 226 and 227. The right disparity vector (DV) densifying unit 235 generates the right pixel disparity vector 237 by scaling every pixel of the generated right disparity vector.
A disparity vector mapping unit 240 generates a mapped disparity vector 241 from the left and right pixel disparity vectors 232 and 237. The mapped disparity vector 241 is a final disparity vector used for synthesizing an intermediate image. The disparity vector mapping unit 240 generates the intermediate image disparity vector 241 by checking a consistency between the left and right pixel disparity vectors and then performing a re-scaling process on the basis of only the consistent disparity vector. An intermediate viewpoint reconstructing unit (IVR) 250 generates the intermediate image 251 from the intermediate image disparity vector 241 and the left and right images 212, 216.
However, the conventional temporal smoothing technology performs the temporal smoothing process uniformly regardless of the type of image. As a result, when a moving picture is static or when a static object exists in the moving picture, the temporal smoothing process is performed smoothly, while when the moving picture is dynamic, the temporal smoothing process may synthesize an unreasonable intermediate image.